Vehicle Driver Behavior Data Collection and Reporting

ABSTRACT

A vehicle safety system performs close-in incipient collision detection that combines high sample rate near-field sensors with advanced real-time processing to accurately determine and report risky driver behavior.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. patent application Nos.62/964,350 filed Jan. 22, 2020 (6917-3); 63/059,727 filed Jul. 31, 2020(6917-7); and 63/117,230 filed Nov. 23, 2020 (6917-8), each of which isincorporated herein by reference.

This application is related to U.S. patent applications Ser. Nos.16/927466 filed Jul. 13, 2020; Ser. No. 16/883,679 filed May 26, 2020;63/030,009 filed May 26, 2020; 62/873,721 filed Jul. 12, 2019; and62/852,841 filed May 24, 2019; each incorporated herein by reference.

BACKGROUND & SUMMARY

Even with all of the safety devices built into vehicles in the UnitedStates, 11% of the over 41 million accidental injuries reported in theU.S. at emergency rooms are due to traffic incidents. In 2017 alone,over 37,000 people lost their lives due to traffic accidents in the U.S.Most of these fatalities occurred in vehicles with fully operationalairbags and where the occupants were using seatbelts. Clearly, existingsafety technology, even when properly utilized, is insufficient toprotect people from serious injury and even death. There is a long feltbut unsolved need to address and overcome these limitations, which everyyear are costing tens of thousands of lives in the US alone and hundredsof thousands of lives worldwide.

Risky driving behaviors that lead to accidents are typically describedin human driver terms, such as weaving in and out of lanes; failure toyield right of way; driving under the influence of alcohol, drugs orother judgment-impairing substance; speeding; not paying close attentionto road conditions; tailgating; and driving while drowsy or distracted.These terms are subjective and can be both inaccurate and misleading. Adriver judged as not paying close attention, for example, may merelyhave made a mistake in reading a road sign. Even speeding is not alwaysdangerous if the driver is merely keeping up with traffic and isotherwise driving in a safe and sensible manner.

What is needed are quantitative measures of dangerous or risky drivingthat are based on measured results.

There have been attempts at building such statistics using variousmeans. Some insurance companies, for example, have begun usingtelematics to monitor driving habits. A telematics system is often asensor cluster comprised a wireless network adapter, an accelerometer, aGPS receiver, an OBD-II interface, and a processor—or sometimes, an appon the driver's smart device equipped with a GPS receiver and anaccelerometer. The telematics system continually gathers information andreports it for evaluation. See for example US20190392529 andUS20200020029. US20190375416 discloses using artificial intelligence toscore driving behavior based on driving data collected from telematicssystems.

While much work has been done in the past, further improvements aredesirable. In particular, it would be highly desirable to gather moreaccurate data that more specifically indicates risky driving behavior.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example overall process.

FIG. 2 is a schematic diagram of an example vehicle.

FIG. 2A shows additional features of an example vehicle.

FIG. 2B shows example non-limiting packaging within a rear view mirrorof a vehicle.

FIG. 2C shows example a non-limiting hardware configuration explodedview of the FIG. 2A rear view mirror.

FIG. 2D shows example non-limiting door clearance detection.

FIG. 3 shows several different hardware architectures for integratingthe sensor system into different hardware communication schema.

FIG. 4 shows an example block diagram of a non-limiting high levelsystem architecture.

FIG. 4A shows an example non-limiting overall functional vehicle blockdiagram.

FIG. 5 shows an example block diagram of a non-limiting vehicle softwarearchitecture.

FIGS. 5A, 5B and 5C show example alternative vehicle processing andstorage arrangements.

FIG. 5D shows an example non-limiting overall system diagram includingplural vehicles, plural processing nodes and plural informationrecipients.

FIG. 6 shows a non-limiting design approach for an example non-limitingtime-of-flight (TOF) sensor.

FIG. 7 shows an example of how an example non-limiting TOF sensor buildsa point cloud that is then converted to an object, which is tracked andused to form a probability of impact.

FIGS. 8A and 8B show example non-limiting approaches for packaging asensor and edge processing assembly into headlamps and tail lamps.

FIG. 9 shows an example non-limiting approach for attaining 360 degreesurround sensor coverage.

DETAILED DESCRIPTION OF EXAMPLE NON-LIMITING EMBODIMENTS

Current insurance automotive measurement schemes typically seek tocorrelate limited data (e.g., acceleration and speed) with accidents.There are at least two flaws with this approach:

1) The defining characteristic of an accident is that the vehicle isstriking something else. The most relevant information is thereforetypically the distance from ego (i.e., the vehicle and driver underevaluation) to the threat, which is currently not being recorded.

2) Accidents are relatively rare events. While some drivers have arecord of accidents leading to a major, fatal collision, many or most donot. Rather than try to correlate data to rare events, one could insteaduse the data to first characterize drivers and/or driving, and thencorrelate the driver/driving characteristics with accidents. In thisway, much larger data groups are provided—meaning much more data whichcan then have meaningful number of accidents per group on which to basestatistically significant correlations.

Example non-limiting technology herein provides an innovative safetytechnology that uses one or more advanced sensors to intelligentlyevaluate and detect risky driving behavior for reporting or otherpurposes. For example, the example non-limiting technology hereinprovides a sensor and processing system which in some embodimentsoperates quickly enough (or is capable of operating quickly enough) toaccurately determine if/when a crash is imminent in the vanishinglyshort time available to act on that knowledge. Such a sensor suite andassociated technology is thus able to also accurately determine whetherdriving behavior relative to another vehicle(s) is risky (e.g., likelyto cause or lead to an accident) even though no accident results fromthe interaction.

Such technology can for example be used to identify or predict riskydriving behavior such as “close calls” or “near misses” of potentialaccidents and crashes in addition to recording detailed data leading toactual crashes. This allows far more robust and effective safetymeasures to be employed as opposed to today's technology winch providesno data regarding near misses and relatively limited data once acollision occurs.

The system in one embodiment can continually operate while the vehicleis in operation. It can log data for real time or intermittent reporting(e.g., over a wireless or other network), making determinations in realtime and display the determinations for driver safety training, and/ortaking preventive actions in the event of a near-certain collision. Inone embodiment, logging of detailed information may be provided in theshort term (e.g., for recovery and analysis in the event of an actualaccident), and determination, selection and/or abstracting of the datamay be provided in the longer term. (e.g., to reduce the massive amountof data that would otherwise need to be reported). Thedeterminations/selections/abstracting take the form of predictions of 1)collision likelihood and 2) potential severity of collision.

Instead of merely (or in addition to) monitoring the “jerkiness” ofdriving behavior or vehicle speed (which may in fact be safe and noterratic) such as through use of a common accelerometer, the examplenon-limiting embodiments sense and measure closing velocities,accelerations and other kinematic measures relating to the physicalrelationships. interactions and relative motion between the ego vehicleand other objects (including but not limited to other vehicles), as wellas distances between the ego vehicle and other objects including othervehicles. The ability to accurately detect and measure relative distanceand motion (e.g., velocity and acceleration) of ego vehicle relative toother objects is a significant enhancement as compared to merelymeasuring the motion of ego vehicle by itself. While cameras can alsobe, used to capture video or other image information of physicalrelationships between ego vehicle and other objects, cameras generatevast amounts of data that typically must be transmitted and analyzed toprovide useful information. While embodiments herein can also usecameras or other ranging sensors (e.g., LIDAR, RADAR, etc.), examplenon-limiting technology herein provides one or more sensors exhibitingextremely fast, responsive motion and/or distance detection thatprovides accurate, unambiguous and potentially compact directindications of how ego vehicle is moving and positioned relative toother, non-contacting objects in its immediate neighborhood. In someembodiments, the sensor suite can detect instantaneous or nearlyinstantaneous closing velocity, acceleration and/or distance between egovehicle and other stationary and/or moving objects, providing accurateinformation from which likelihood or risk of collision can be inferredor predicted. Such information can be collected over time to predict ormeasure risk of future accidents (and their severity) associated with aparticular driver(s), a particular vehicle or kind/type/model ofvehicle, a particular driving environment (e.g., geographical, weather,time of day, season, etc.), particular driver age or othercharacteristics (e.g., teens vs. older people, elderly drivers, driverswho are out late, etc.), particular driving routes/cities/highway types,or any combination or subcombination of these.

In one example embodiment, these data feed into two or more differentalgorithms for real-time or other processing: 1) a Collision ProbabilityEngine (CPE) and 2) a Collision Severity Calculator (CSC). The CPE usesthe kinematics of the ego vehicle and the sensed kinematics of threatobjects to calculate all (or likely) physically possible outcomes. Fromthat, it can calculate the probability of a collision. As the underlyingalgorithms are relatively simple, they may be calculated in real time oncomputation platforms of reasonable size, power and cost for theautomotive market. The CSC operates in similar fashion. The potentialphysically-realizable paths that result in a collision are evaluated asto the potential threat they pose should a collision actually occur. Forexample, the threat posed by having a large object impact the driver'sdoor would be judged as greater than the threat of sideswiping a muchsmaller object.

The active sensor suite can be used to report events such as near missesor even actual collisions in real time or on a delayed basis foranalysis and action. For example, if real time reporting is used, afleet dispatcher can check with a driver after a collision is reportedto see if assistance is needed or a parent could call a teen driver tofind out if everything is okay. Based on any reporting scenario, aninsurance company can modify insurance coverage of a habitually riskydriver or require the driver to obtain driver safety education.

The active sensor suite can also distinguish with high accuracy betweensafe driving and risky driving. A driver may be weaving in and out oftraffic, for example, to avoid aggressive drivers sharing the road,which mere acceleration-based telematics of vehicle motion cannotreveal. The example non-limiting active sensor suite can detect suchexcellent driving skills and provide rewards such as good drivingdiscounts or letters of congratulation for avoiding accidents inhazardous conditions instead of penalizing an excellent driver fortaking successful evasive action to avoid accidents in challengingdriving environments

In some embodiments, processing can be performed locally in real time togive drivers feedback on how they are driving. In one embodiment, anadditional example display could be onto a projected holograph or ontothe scene projected on the windshield (e.g., augmented reality in thestyle of a heads up cockpit display in a fighter jet) and/or projects onthe side or rear view mirrors, dashboard, or other surface visible tothe driver. In other embodiments, the display could be a compact drivingscore value (e.g., 1 to 10, A through E, etc.) that indicates how safeor risky the driver is driving.

In some embodiments, data can be logged and reported in real time and/orintermittently. Raw collected data can be reported to a processor in thecloud which performs the analysis; or the data can be fully processedand results are reported; or the data can be partially processed andresults can be reported along with some or all of the underlying rawdata. In some embodiments, results can be reported and underlying rawdata is reported upon request/demand.

Reducing scenarios to probabilities allows calculated probabilities tobe added to the telemetry stream, as such data consume little bandwidth.Such compact telemetry is much more compact than video or raw TOF sensordata.

In addition to probabilities leading to a collision, it is possible totell when an accident is actually occurring or has actually occurredwith very high confidence.

Data analysis is in one embodiment expanded from probability ofcollision to listing collision parameters that can be correlated withfault. For example, the fault level for a situation when ego is brakingbecause he was cut off suddenly would be lower than if ego was brakinghard because the car in front was braking normally; the fault levelwould be higher for a side fender impact if ego were maneuvering towardsanother car that was traveling in a straight line, and so forth.

One example non-limiting embodiment of a system is packaged as a dashcam(era). The system can in one example implementation be packaged withthe CWTOF with 120° FOV, a cellular uplink (and/or Bluetooth connectionto user's phone), IMU for precision ego motion analytics, GPS and aruggedized nonvolatile memory unit “black box.” This “black box” can berecovered and used for crash reconstruction, by providing kinematic datafor events just before, during and following the collision forpre-crash, post-crash and other applications.

In other embodiments, if a high bandwidth connection is available, thesystem can automatically send a burst of data during or immediatelyafter a collision to another computer in the cloud in order to provideclose to real time indications of what is happening or has justhappened. While in some embodiments only telemetry-level data would beuploaded automatically, the system would have ample storage to save,say, 25 of the latest events in a manner that would preserve the dataeven in the event of a catastrophic accident. A data recipient such asan insurer, employer, parent, rental car agency, law enforcement, healthcare provider, etc. could then (with appropriate authorizations and/orconsent) command the system to upload or otherwise obtain actual sensoroutput for specific events of interest.

In one embodiment, as a CWTOF sensor, the system collects both B&W videoand the distance (Z)-map. Both can be stored, even though the system insome embodiments uses only Z-map data for its probability calculations.

The system works day or night and quite well in snow, rain or fog. Thesystem in some embodiments could measure many atmospheric conditions, soas to measure how safe the driver is being in inclement weather. This ispossible by for example measuring the return off of retroreflectiveheadlamps or through the use of other sensors such as moisture sensors,ice sensors, temperature sensors, windshield wiper operation monitoring,etc. In the case of retroreflective headlamps, if the system receives anunusually low return on a large number of samples, then that probablycorrelates well with atmospheric extinction coefficient, which itselfcorrelates with weather conditions.

The system in some embodiments can record ambient light levels to detectlow visibility situations or otherwise measure visibility (e.g., it caneven tell if the sun was in the driver's eyes during an event).

Since the system in one embodiment can output a very bright light in theNIR, lane markings, which are painted with special retroreflectivepaint, show up with excellent contrast. If the IMU (inertial measurementunit), for example, is indicating weaving-like behavior, this could beflagged in the telemetry stream, and the camera could record the eventfor later visual inspection.

Other features include:

Determine if an unreported crash happens

Determine the density of average traffic (e.g., driving in a congestedarea is far more conducive to an accident versus rural driving)

Parking behavior

The number of times a vehicle is in close proximity to another movingvehicle (e.g., two moving vehicles get to within a predetermined closedistance such as say 10 feet while driving over 50 mph, etc.)

Some embodiments provide an “insurance” algorithm that identifies riskybehavior and sets triggers. This “insurance algorithm” can beprogrammable/adjustable so that different insurance companies cancustomize the results differently. Such a risk assessment algorithm maydetect the probability of collision, and may also apportion cause. Forexample, if there is a near miss from a head-on collision, and thethreat car suddenly maneuvered to a collision course, then the faultwould lay mostly with the threat car. If on the other hand, it was egothat maneuvered into a collision trajectory, then ego is at fault. Suchalgorithms may use a lot of data collection and then correlate recordeddriving behavior with outcomes (e.g. driving records) over time. Thereis a big difference between someone who weaves in and out of traffic andcontinually cuts people off as compared to another driver who drivessteadily but in a bad traffic environment, even if both see the samenumber of above-threshold probability events. Some embodiments can alsosend video of an accident plus/minus 5-10 seconds for insurance claimshandling, as well as all kinds of triggered data.

Near miss and/or actual collision data can be sent to vehicle OEMs toenable them to understand an actual collision so they can improve uponseverity reduction—not real time, but after the fact. From an accidentreproduction point of view, it will be very valuable to have that data.

A collision causality model may be used so that in the event of acollision, the system can tell collision speed, severity, and who is thelikely perpetrator that caused the incident. Such an analysis wouldassist or remove the need for more insurance company claim adjusters.

In one embodiment, a combined solid state LIDAR and ToF sensor with aSoC (system on a chip) doing sensor fusion and normal precrashfunctionality plus ADAS may be provided in the various use casesdescribed herein.

Fusing ADAS long-data, with close-in high speed systems may improveprobability engine results.

Employing the same or similar techniques may improve the severityestimation.

Fuzing a long-range, slower (but more accurate) sensor with a close-insystem may be provided in some embodiments. The longer range systemcould be LIDAR or Doppler radar since that yields a very accurate speedtowards the vehicle. Some embodiments may use the data to calibrateoutputs on the fly and avoid calibrating to something that is lessaccurate. Varying the CWTOF parameters on the fly may be used to extendrange, while still allowing high speed sampling for close in objects.

Active Sensor Suite

An active sensor suite may use high-speed, near-field optics andembedded computing or other example non-limiting high speed sensing,tracking and processing technology. Example non-limiting embodiments candetermine precisely when and where (e.g., in which direction) acollision is going to occur, buying precious time to respond. With thisforewarning, robust countermeasures can have sufficient time to engage.This may include for example external airbags that cushion the blow topedestrians or protect vehicle occupants from lethal side impacts.Larger and thus far more effective airbags can be employed that bothinflate at slower speeds and serve to protect occupants on all sides.The vehicle itself can improve its crashworthiness by adjusting thesuspension and internal cab features such as the steering column ormoving seats.

In other embodiments, there is no or may not be any deployment of acountermeasure. For example, in some embodiments, no countermeasure ispresent and information is collected, stored and forwarded for otherpurposes (e.g., insurability/risk evaluation). In other embodiments, thesystem feeds collision probability, time, severity and impact locationto another system, and the other system uses that information to decidewhether to control the countermeasure to deploy.

In example non-limiting embodiments, a relevant time window for thetechnology herein to operate is shortly before an actual or potentialimpact—i.e., from a time when the driver of the vehicle can do little toavoid or minimize the impact should it occur. Such time window imposesreal time and near field constraints that lessens the intent of thedriver(s) and reduces the collision problem to more deterministicfactors such as Newtonian physics. In some embodiments, rather thanavoiding a collision, the vehicle automatically and swiftly prepares fora collision beginning at a time instant so soon before the collisionactually occurs that the collision is highly predictable andunavoidable. Such a time window can begin for example at one second, orone-half second (500 milliseconds), or 0.25 seconds (250 milliseconds),or 0.2 seconds (200 milliseconds), or 0.15 seconds (150 milliseconds),or 0.1 seconds (100 milliseconds) or 0.05 seconds (50 milliseconds),before instant of impact. A sufficiently short time removes all relevanthuman behavior, allowing crashes and near misses to be evaluated onpurely quantitative terms. Some have measured the average time forhumans to react to a visual stimulus at 0.25 seconds (“human reactiontime interval”). It is unrealistic to expect the human driver of avehicle to react and respond in less time than the human reaction timeinterval, and the principles of physics may dictate that based on theconditions (i.e., motion of the ego vehicle, road conditions,potentially motion of another vehicle, etc.) that are determined toexist at a particular instant in time, an accident or collision is orwould have been unavoidable within the human reaction time intervalsubsequent to that particular instant in time.

Because deploying countermeasures such as air bags will often have theeffect of distracting or hindering the driver from being able to seeand/or operate vehicle controls effectively, deploying countermeasurestoo early could have deleterious effects of (a) preventing the driverfrom taking evasive action to avoid a collision or minimize its impactand/or (b) deploying a countermeasure before an impact is a certainty(and thus might still be avoidable). However, waiting until an impact isdetected (as most modern systems do) wastes precious milliseconds beforean imminent crash that could be used to deploy a lifesavingcountermeasure.

Example non-limiting technology herein uses highly reliable very fastacting detectors that when combined with the appropriate softwarestatistically have very few false positives within the short time windowbefore a collision (i.e., they can be counted on to reliably detect witha high degree of certainty that a crash is both imminent andunavoidable) to deploy countermeasures such as air bags or otherlifesaving techniques.

The example non-limiting technology herein fits well with the ADAS andAV architectures already being considered in the industry and in somecases even fielded across the industry. While these technologiesthemselves will provide further safety in crash avoidance, vehiclecrashes, sadly, are projected to continue increasing globally beforeeventually declining decades from now. The example non-limitingtechnology herein holds forth the promise of making the roads safer inthe very near future, and by a substantial margin. Deployed worldwide,the technology promises nothing short of a revolution in automotivesafety that eclipses even the advances enabled by seatbelts and thecurrent generation of airbags.

While the automotive industry is focused on active safety (e.g. ADAS),little effort is made towards passive safety technology (e.g. seatbeltsand airbags). Where ADAS focuses on driver convenience in addition tosafety, it accomplishes this by enhancing emergency maneuvers of the,vehicle via means such as active braking and lane intrusion warnings.Example non-limiting embodiments herein, on the other hand, focus onsafety at the last fraction of a second where no amount of vehicularmaneuvers or braking can avoid an imminent or potential collision.

Example non-limiting technology herein provides sensors and systemcontrol architectures that are capable of collecting and processing manlarge quantities of data every second. This allows an extremely accuratecountermeasure solution to be fully deployed in just a few millisecondsor less. Example non-limiting technology herein is able to accuratelypredict the collision of a vehicle with another vehicle or object andthen launch one or a suite of countermeasures, such as advanced airbags,to protect occupants in ways heretofore considered impossible.

Example non-limiting embodiments provide a sensing and advancedprocessing system that can predict an imminent or potential crash withassurance. Adding even 50 milliseconds would roughly double the reactiontime. While early simulation experiments indicate that as much as 250milliseconds of warning of an imminent unavoidable collision isachievable, even a fraction of that would revolutionize the passivesafety market. The non-limiting embodiment can also operate up to asecond ahead of a crash for selected types of countermeasures and inselected scenarios. With the addition of more time, passive safetysystems can do much more to prepare the vehicle for a collision: deployexterior airbags, adjust the suspension, and inflate larger interiorairbags at slower speeds. These combined measures translate directlyinto reduced injuries and fatalities due to the crash and certainly dueto the airbag itself.

Additional example non-limiting features and advantages include:

High Speed—threat tracks leading predictions of collision and severityare developed in a very limited amount of time

Near Field—Removes human intent (lateral or in-line) and is based purelyon the physics of the objects (e.g., momentum under Newton's laws thatcannot change a crash scenario significantly no matter what the humanoperator or an obstacle such as another vehicle might do or attempt todo)

Fast Real Time Processing—One example embodiment uses massive parallelprocessing of sensors feeding into distributed FPGAs (field programmablegate arrays) or similar computation capabilities.

Takes the velocity(ies) and/or acceleration(s) of objects into accountwhen determining when a collision is imminent or will result in a closecall.

Example non-limiting embodiments perform close-in collision detectionthat combines high sample rate near-field sensors with advancedreal-time processing to accurately determine imminent threats and thelikelihood of a collision. This allows applicable countermeasures to bedeployed in some embodiments based upon the (high) probability of acollision, while also taking into account the type of threat, and theimpending impact's location on the passengers of the vehicle. This newapproach will soon transform the passive safety market to greatly reduceinjuries from automotive crashes and safe lives.

For example:

Suppose one is driving a vehicle moving at 65 mph (30 m/s) and anoncoming vehicle is moving at 65 mph (30 m/s). The combined relativespeed is 130 mph (60 M/s).

Let's assume the system needs 100 milliseconds advance notice to deploythe external airbags and other new passive safety features.

Therefore, the vehicle would have to determine an imminent threat whenthe oncoming vehicle is 6 meters away, about one vehicle length.

Example non-limiting technology herein waits until the last possiblemoment (or within a time window that includes the last possible moment)to automatically deploy a countermeasure when no human or machinemaneuver can change the outcome (e.g., whether or not a crash occurs).In other embodiments, where no countermeasure is deployed, the systemcan collect data during this short time period that other systems arenot able to sense or sense reliably in order to determine the cause ofan accident and who is at fault and/or to detect whether a potentialcollision was averted by luck, by skill, or for some other reason.

Given the extreme rapidity of the required response in some embodiments,an example non-limiting system may use a sensor which can sample from atleast 200 Hz to 500 Hz or faster. In one example non-limitingembodiment, the data from this sensor feeds into a high-speed processorto determine the vehicle trajectory with the accuracy of e.g., 10 to 20centimeters. From this, the impact point on the vehicle is estimatedalong with the likelihood of serious collision. The timing and sequenceof countermeasures, if any, are calculated and controlled. All of thishappens (and needs to happen) in just a few tens of milliseconds inexample non-limiting embodiments.

Additional non-limiting features include:

A system comprising: a sensor that senses an object; a processoroperatively coupled to the sensor that determines, in response to objectsensing, an imminent collision with the sensed object before thecollision occurs; and a countermeasure operatively coupled to theprocessor, the processor controlling the countermeasure to deploy inresponse to determining the imminent collision.

The sensor may comprise a non-contact sensor.

The processor may determine an imminent or potential collision during atime window within the range of 250 milliseconds, 200 milliseconds, 150milliseconds, 100 milliseconds, 50 milliseconds and/or 0 milliseconds(this is still useful since it would assist the standard inertialsensors to react more quickly and precisely) before a collision occurs.It may also determine a lower-probability of collision at longer timessuitable for some types of countermeasures.

The countermeasure may comprise an air bag, a seat belt tensioner, anexternal pedestrian air bag, an adjustable seat, an active suspensionsystem or other countermeasure. Or in some embodiment there may be nocountermeasure and the purpose of the system is simply to gather datafor current and/or later analysis.

The sensor may comprise a time-of flight sensor, stereoscopic sensor orother.

The sensor may sense parameters relating to the range, xy position, rateof closing and direction of closing of the object.

The sensor may be disposed on a moving object or on a stationary objectthat tracks the moving object. For example, in one embodiment, thesensor is mounted at a stationary position that is subject to collisionwith a moving object. The sensor may sense an imminent collision, anddeploy a countermeasure(s) (from/on the moving object, from/on thestationary position, or both) to minimize potential damage as a resultof the imminent collision.

The processor may track the sensed object as it closes, and output theerrors associated with the track

The processor may use the track and the errors associated with it todetermine the probability of an imminent collision.

In one embodiment, the system includes an actual collision detector, andthe countermeasure is deployed in response to the actual collisiondetector detecting an actual collision.

In an alternative, a system comprises a sensor that senses a closingobject; and a processor operatively coupled to the sensor thatdetermines, in response to object sensing, an imminent collision withthe sensed object before the collision occurs.

A method comprises sensing an object; determining, in response to objectsensing, an incipient collision with the sensed object will occur in 250milliseconds or less.

The sensing may be performed continually, and the deploying may occur atleast in part after the collision occurs.

Example Non-Limiting System

Example non-limiting embodiments operate in three stages as shown inFIG. 1. First, the system senses an imminent or potential collision orother unsafe situation (20). This is accomplished in one examplenon-limiting embodiment by using a low-cost, high data-ratetime-of-flight (TOF) or other type of fast sensor looking out no fartherthan for example 20 meters. Such sensing may be performed continually.Several sensors may, for example, be placed around the perimeter of thecar housed in headlight and taillight fixtures such as shown in FIG. 8A,8B. Other suitable sensors may include or comprise a stereoscopic sensoror a short-range radar sensor or an imaging acoustic sensor or a visiblelight sensor or an ultraviolet or infrared light sensor or a LIDARsensor. Together, the sensors search the scene for elements that aremoving relative to the vehicle. These elements are then defined asobjects.

Second, the objects are tracked over time (22). The track is plottedwith an error band that allows the probability of an impact to becalculated. In general, the errors tighten as the object approaches orcloses. In addition, the size, speed and likely hit location of theobject are used to estimate the potential severity of any collision.

In some non-limiting embodiments, the object must meet the criteria tobe declared a valid threat:

-   1. It must be judged as having a sufficiently high severity as to    present a threat. For example, a glancing blow to the rear fender    would be less likely to be gauged a threat than a head-on collision    (at least for protection of the vehicle occupants).-   2. The object must have a high likelihood of impacting the vehicle.-   3. The object must be large enough to warrant countermeasures to be    launched. For example, a short, small box may not pass the    threshold, while a much taller, telephone-pole sized object would.

Example non-limiting embodiments that have countermeasures may err onthe safe side of not launching countermeasures if the determination ofan imminent collision is uncertain. For this reason, a normal passivesafety system as initiated by an IMU can also remain fully functioningas a back-up safety system to deploy countermeasures when impact isdetected. The operations of the IMU will be enhanced by the informationavailable from the track and severity estimations.

FIG. 2 shows is a visual image of an example non-limiting a stand-alonesystem in a passenger car 50, although the technology herein can beapplied to any kind of vehicle or other moving object. It shows thesystem's ECU (electronic control unit including at least one processorand associated memory) 52 taking in and processing the vehicle's sensorfeeds 54. The system can be fully integrated with the vehicle systemsincluding the IMU-based (e.g., accelerometer) airbag and seatbelttensioner control system.

FIGS. 2A-2D show additional example applications for the same sensorsuite and associated system including for example frontal pre-crash andclearance detection, curb detection, side impact pre-crash detection,gesture control (FIG. 2A). FIG. 2D shows example door clearancedetection as a secondary or auxiliary function if the sensor is disposedon the side of vehicle 50 such as within a rear view mirror. FIGS. 2Band 2C show example sensor packaging for sensors within rear view mirrorassemblies (FIG. 2B) providing side mounted multidirectional arrays(FIG. 2C), as discussed below.

In more detail, FIGS. 2A & 2D show other example use cases for the FIG.2B/2C sensor configuration (and which can also be applied to othersensor configurations such as shown in FIGS. 8A/8B) including but notlimited to:

-   -   Frontal PreCrash & Clearance Detection—Advance passive safety        system such as intelligent airbags and seatbelts by detecting an        imminent impact 50-60 milliseconds prior to contact.    -   Side Impact Precrash: Advance passive safety system such as        intelligent airbags and seatbelts by detecting an imminent        impact 50-60 milliseconds prior to contact.    -   Automatic Door Opener Clearance Detection—determines ground and        object clearance for automatic door openers and manual door        opening.    -   Curb detection—Identify curbs forward, side, and aft of the        vehicle for automated parking or park assist. Especially useful        for independent rear steering such in the Hummer EV or Mercedes        S-class.    -   Gesture Recognition—The driver with or without a key fob can        make a gesture to control various aspects of vehicle operation        such as rolling-up the windows after parking car, turning on or        off the vehicle lights, locking or unlocking the vehicle, etc.

FIG. 3 shows more detail for example non-limiting hardware integrationapproaches. In this case, “butler service” refers to features such asgesture recognition or sensing that the car is parked too close toanother for safe egress. In the light ring network example, the precrashsensors (shown here as circular arrays of emitters) are linked using aprivate gigabit Ethernet connection. The connection to other vehicleprocessors and apparatus is via the standard vehicle CAN bus. Thissupports precrash and butler services, but has insufficient data rate tosupport most ADAS and AV applications. In the ring network example, thering network architecture is similar to the light ring network, exceptthat it uses a vehicle-level gigabit Ethernet to send data to ADASprocessors which can be enhanced to provide sensor support functions.However, there is still insufficient bandwidth to support most AVapplications. In the star network example, the star network usesmultiple gigabit Ethernet connections to communicate to a centralprocessor, thereby having sufficient bandwidth to pass full-frame,high-speed video that is required by many AV applications. The starnetwork example shown includes an additional proprietary processingboard such as shown in FIG. 6 to provide a processor for sensor imaging.Any combination of these and/or other network topologies may be used.

FIG. 4 is a block diagram of an example non-limiting high-level systemarchitecture 100 that includes vehicle controls 102, a countermeasuredeployment system 104, an advanced driver assistance system (ADAS) 106and sensors 108. While the countermeasure deployment system would not beused in near misses, it would be activated in cases where there is anactual collision in some embodiments.

In the example shown, the vehicle controls 102 including steering 152,braking 154, powertrain 156 and command arbitrator 158 may beconventional and the same ones commonly used on any or many modernvehicle(s). The RDU 104 may include a conventional inertial measurementunit (MU) 168 including inertial sensors such as accelerometers and/orgyrosensors but in this case its output is provided to a commandarbitrator 170 that controls deployment of various countermeasuresincluding seat belt restraining system 160, interior airbags 162,exterior airbags 164 and other countermeasures 166. In the exampleshown, the seatbelts 160 and interior airbags 162 can be enhanced toaccept control from the IMU 168 (indicating a collision has occurred) orvia the command arbitrator from ADAS 106 (indicating a collision isimminent but has not yet occurred) so the seatbelts tighten and theairbags deploy, and these countermeasures can behave differentlydepending on which system is actuating their deployment. Additionalcountermeasures 164, 166 not found on a conventional vehicle (e.g.,pedestrian airbag system, bumper shock absorber stiffener, framestiffener, automatic wheel turning, seatbelt tightener, etc.) may bedeployed primarily in response to control from ADAS 106 detecting acollision is imminent. For example, one possible countermeasure couldinclude last second chassis actions to mitigate the impact such as e.g.,braking only with one brake (e.g., the right front brake or the leftfront brake, depending on the direction of expected impact) to helppivot the center of gravity to reduce the force of impact.

In the example shown, the ADAS Active Safety block 132 may be enhanced(as described below) and ADAS 106 may be provided with new functionalityincluding a 3D corrected object view block 174, high speed objecttracking 176, threat assessment 178 and fire control solution 180.Object tracking 176 is based on the 3D corrected object view 174, whichin turn receives input from a sensor such as a very fast time of flight(TOF) or other sensor that can detect shape and direction. Sensor 182can for example be non-contacting and based on technologies such asoptical, electromagnetic, sound-based ultrasonic), laser-based,radar-based, or any combination thereof. In one example embodiment, thesensor 182 (also referred to in the specification and the drawings as“TrueSense” or “TOF” or “time of flight” sensor) is as described in U.S.patent application Ser. Nos. 16/927466 filed Jul. 13, 2020; Ser. No.16/883,679 filed May 26, 2020; 63/030,009 filed May 26, 2020; 62/873,721filed Jul. 12, 2019; and 62/852,841 filed May 24, 2019; eachincorporated herein by reference as if expressly disclosed herein forpurposes of disclosing/enabling/supporting FIG. 2C, FIG. 4 block 182,FIG. 6, and for all other purposes.

The 3D corrected object view 174 is used by threat assessment 178 toassess threat of a likely collision. If threat assessment 178 determinessuch a threat exists, it records it as a near miss if the threat did notcollide and a collision if it does. In some embodiments, the threatassessment information is communicated to cloud storage via thecommunications system 190. FIG. 4A shows a more detailed exampleprocessing block diagram in which sensor 182 and IMU 168 provide outputsto determine both ego vehicle trajectory 186 and probability ofcollision (POC) and time to collision (TTC). The object trackingdescribed above (block 176) is responsive to the output of sensor 182. Aperturbed trajectory 188 is calculated to provide a perturbedprobability of collision (PPOC) and a perturbed time to collision(PTTC). See e.g., Terry, “Collision operator and long-time behavior of aperturbed two-body problem”, Celestial Mech.; (Netherlands); Volume: 23pages: 119-130 (Sep. 1, 1981); Astarita et al, “Trajectory Perturbationin Surrogate Safety Indicators” Transportation Research Procedia Volume47, Pages 393-400 (2020), each incorporated herein by reference.

At the core of one example non-limiting solution is an Edge CrossCorrelation Engine (ECCE) or “edge processor”. With ECCE, the examplenon-limiting system can perform rapid, direct 3D object definitionswithout the need for complex calibration or resorting to costlyartificial intelligence (AI) methods. Once objects are formed, they canbe readily tracked, which leads to the prediction of whether or not acollision will occur with its commensurate likelihood.

FIG. 5 shows an example non-limiting software block diagram of asoftware architecture that implements an ECCE described above. Thisarchitecture includes a kinetic microcontroller 202 that executesinstructions stored in non-volatile memory to very rapidly capture andprocess data from multiple time-of-flight (TOF) sensors 182. A processor(CPU) 204 implements detection, tracking and collision prediction byperforming detection algorithm(s) 206 and processing algorithms based onsimulation 208 (which may be used for tracking and collisionprediction). The processor 204 may provide real time visualization 210and output messaging via communications 190 to cloud storage. The sensorand/or edge processor 204 can be integrated in headlamp, tail lamp,rearview mirror housing, A pillar, B pillar, front grill, front bumper,rear bumper or trunk housing of a vehicle.

Example Processing Timing and Placement

In one example non-limiting embodiment, all processing takes place onthe ego vehicle. In another non-limiting embodiment, little or noprocessing takes place on the ego vehicle and most or all processinginstead takes place at a remote location such as in the cloud. In otherembodiments, some processing takes place on the ego vehicle and someprocessing takes place in the cloud. In any of these cases, theprocessing can be performed in real time, in close to real time, or on adelayed basis.

If the ego vehicle includes a countermeasure, it is desirable to performat least imminent collision on the ego vehicle in real time. However,some embodiments do not deploy a countermeasure or do not use certainprocessing in order to deploy a countermeasure. In such embodiments,processing can be delayed to close to real time or not real time.Furthermore, depending on the communications bandwidth available, it maybe desirable for the ego vehicle to transmit to a remote device(s) allof the data it collections, some of the data it collects, or little ornone of the data it collects. For example, given the speed at whichsensors such as 182 can acquire data, such sensors can stream a hugeamount of data in a very short time. It may be impractical orunnecessary to communicate such a firehose of data to a remote computer.Instead, the system on board the ego vehicle can analyze, filter,abstract, select from, condense, compress, parameterize, or otherwiseprocess the sensor data to provide it in a more useful and/or condensedform for use by the remote computer. In the meantime however, it may beimportant or helpful for the ego vehicle to collect and preserve amaximum amount of data from just before, during and immediately after anactual collision for use by insurance companies, law enforcementauthorities, courts and other decision makers.

A non-limiting example process flowchart is shown in FIG. 5A. Precrashsensors such as the TrueSense TOF sensor 182 collect data at a high datarate. Much of these data are processed at the edge, conducting functionssuch as converting phase data into point cloud data, minimizingdistortions, smoothing temporal fluctuations, demodulating encoded dataand so forth. Other sensors that also contribute to eventual behavioralanalytics, such as a driver camera C, may also be included, and thesemay also have some kind of edge processing, although that is not alwaysrequired. Processed data from both source types are then communicatedvia a high data rate bus 305 such as IEEE 802.3 Ethernet networks. Dataare streamed to 1) data recorder(s) 308, 2) precrash processor(s) 312and 3) behavioral analytics processor(s) 314. The data recorder 308would normally only record data for some set amount of time (such aslast hour of driving for example) or more likely as related to specifiedevents from the precrash processor 312 such as a near-miss or passing aclose object at extreme speed. The precrash processor 312 in thisexample processes data in real-time so as to provide the appropriateprecrash inputs to safety actuators such as smart seatbelts. In thisexample, precrash data are also passed to a real-time behavioralanalytics engine 314 that determines whether the driver is performingwithin normal safety limits or not. By processing data on-board, thebehavior analytics engine 314 can then communicate relatively lowbandwidth data to the cloud 322 such as some safety score or perhaps ascore with small segments of sensor data. Further post processing can beperformed on data in the cloud.

A further non-limiting example process flowchart is shown in FIG. 5B.Precrash sensors such as the TrueSense TOF sensor 182 collect data at ahigh data rate. Much of these data are processed (302) at the edge,conducting functions such as converting phase data into point clouddata, minimizing distortions, smoothing temporal fluctuations,demodulating encoded data and so forth. Other sensors that alsocontribute to eventual behavioral analytics, such as a driver camera,may also be included, and these may also have some kind of edgeprocessing (304), although that is not always required. Processed datafrom both source types are then communicated via a high data rate bus306 such as IEEE 802.3 Ethernet networks. Data are streamed to 1) datarecorder(s) 308, 2) precrash processor(s) 312 and 3) behavioralanalytics processor(s) 314′. The data recorder 308 would normally onlyrecord data for some set amount of time (such as last hour of drivingfor example) or more likely as related to specified events from theprecrash processor 312 such as a near-miss or passing a close object atextreme speed. Since data are being uploaded to the cloud 322 fromrecorded data 308, the behavioral analytics engine 314′ in this casedoes not need to be real time. It may also be somewhat simplified inthat its primary function is to decide which data need to be stored,rather than providing final safety scoring itself. As before, theprecrash processor 312 in this example processes data in real-time so asto provide the appropriate precrash inputs to safety actuators such assmart seatbelts. The data is communicated to the cloud 322 asconnections and available bandwidth permit. It could, for example, be astreaming 5G service or a less regular WiFi connection when the vehicleis parked in the owner's garage. The amount of data to be communicatedis largely a function of the sophistication of the on-board behavioralanalytics engine 314′ and client requirements.

A further non-limiting example process flowchart is shown in FIG. 5C.Precrash sensors such as the TrueSense sensor 182 collect data at a highdata rate. Much of these data are processed (302) at the edge,conducting functions such as converting phase data into point clouddata, minimizing distortions, smoothing temporal fluctuations,demodulating encoded data and so forth. Other sensors that alsocontribute to eventual behavioral analytics, such as a driver camera C,may also be included, and these may also have some kind of edgeprocessing (304), although that is not always required. Processed datafrom both source types are then communicated via a high data rate bus306 such as IEEE 802.3 Ethernet networks. Data are streamed to 1)precrash processor(s) 312 and 2) remotely located behavioral analyticsprocessor(s) (328). Note that a data recorder could also be included asa backup, but it would not strictly be required since all or most dataare being streamed to the cloud 328 in this example. The precrashprocessor 312 processes data in real-time so as to provide theappropriate precrash inputs to safety actuators such as smart seatbelts.Both processed precrash data and high-bandwidth sensor data are combinedand transmitted to the cloud 328 to be stored and post processed forbehavioral analytics and other purposes.

FIG. 5D shows an example overall system that includes many such vehicleseach of which include the capabilities described above. For example, inthe example shown, a plurality of cars 50, a plurality of boats 1050 orother watercraft, a plurality of trucks 1052, a plurality of busses 1054and a plurality of personal flight vehicles 1056 are each monitored witha system of the type shown in FIGS. 4-5C. Any type of moving objectincluding but not limited to any time of vehicle (manned or unmanned)may be so monitored. The various vehicles communicate via FIG. 4 block190 the data as described in FIGS. 5A, 5B, 5C to one or more remoteprocessors 1062 in the “cloud” via one or more telecommunicationsnetworks 1060 which may be based on cellular telephone, wifi, wimax, orany other suitable wireless networking or communication technology. Theprocessors 1062 then selectively process the received data, filter theprocessing results, and provide the filtered processed results todifferent ones of service providers 1064 such as insurance companies,employers, fleet operators, vehicle maintenance companies, etc.;authorities such as courts, law enforcement agents, highway authorities,etc.; families 1066 such as parents of teen or adult children of elderlydrivers; apps 1070 such as smart phone apps operated by the driversthemselves or other authorized participants; and others. Suchcommunications in many embodiments will in all cases authorized toprotect privacy. For example, no communication is made to lawenforcement agencies or courts unless there is consent or a warrant hasbeen issued.

Driver Skill/Behavior/Risk Determination

Once the FIG. 5D system correlates driver behaviors with accidents, oneonly needs to characterize an individual driver based on thecharacteristics—something that could be done in a short amount ofdriving time. Also, if drivers know what the characteristics they'rebeing judged on are, and which are affecting their insurance rates, theycould get real-time feedback on the car dashboard or on an app on theirsmart phones, if they wanted such, so that they could modify theirbehaviors to reduce their rate.

Drivers could be categorized on different scales—e.g., attentiveness,aggressiveness, distancing, and skillfulness—to name a few. Behaviorscan be characterized in terms of relations to neighboring vehicles.Doing 80 mph on an empty highway may be illegal, but it may notparticularly dangerous or risky if visibility is good, the road is dryand no other adverse considerations apply. Doing 80 mph on a welltrafficked road where most others are going 65 mph can be exceedinglydangerous, even reckless. The difference is ego's behavior relative toneighbors rather than ego's own behavior.

Processors 1062 and/or 300 may, by executing instructions stored innon-transitory memory, determine, calculate or otherwise ascertainexample driver behavior evaluation factors or characteristics including:

Attentiveness: Measure of driver attention to the road based on trafficcircumstances. Main measures may include:

-   -   Driver focus as measured by dash cam(era) or other sensors    -   Ego distance to neighboring vehicles    -   Speed    -   Maneuver status (straight, turning, accelerating, stopping,        etc.)    -   Neighbor status (no neighbors, neighbor in adjacent lane in        blind spot, neighbor in opposing lane moving at a certain speed        or greater, neighbor move orthogonally to ego (e.g. an        intersection, etc.)

Correlations may include:

-   -   Driver focus correlated with distance, speed, maneuver status        and neighbor status both individually and covariantly    -   Aggressiveness

Measure of driver maneuvering; Main measures may include:

-   -   Speed    -   Velocity    -   Acceleration    -   Relative speed    -   Relative velocity    -   Relative acceleration    -   Ego distance to neighboring vehicle(s) or other objects    -   Maneuver status    -   Neighbor status

Correlations may include:

-   -   Distance from neighbors vs. speed and relative speeds to        neighbors    -   Distance from neighbors vs. acceleration and relative        acceleration and/or speeds to neighbors    -   Space between front/back neighbors during lane change and        relative velocities (e.g. cutting neighbors off)    -   Number of distinct maneuvers over time correlated with number of        neighbors per maneuver (e.g. weaving through traffic)    -   Distancing—built on assumption that you can't hit (or be hit by)        something if nothing is nearby

Measure of time ego spends in close proximity to neighbors. Mainmeasures may include:

-   -   Speed    -   Ego distance to neighbors    -   Relative speeds of neighbors

Correlations may include:

-   -   Ego speed adjustments vs neighbor location and speed (is ego        trying to maximize distance?)

Skillfulness—Measuring things known to take some skill. Main measuresmay include:

-   -   Ego distance to neighbors    -   Maneuver status    -   Distance between cars before parallel parking    -   Distance between cars before back in or front in parking

Correlations may include:

-   -   Number of maneuvers taken to parallel park vs. distance between        cars    -   Speed of maneuvers (e.g. fluidity)    -   Final distance from curb and neighbors    -   Number of maneuvers taken in front/back in parking    -   Speed of maneuvers    -   Final distance from neighbors.

Example Sensor 182 Configuration

As shown in FIG. 6, one example film-limiting embodiment takes acontinuous wave (CW) TOF sensor that has been designed to sample at veryhigh speed (up to 500 samples per second). The 101 imager and processorprinted circuit board (PCB) is connected to the 105 emitters (in thisparticular case a linear array of 8 emitters) via a 103 flexibleconnector, typically via a gigabit Ethernet protocol. The power,external interconnects and auxiliary components are housed in 107. FIG.2C is an exploded view of the FIG. 2B rear view mirror assemblyproviding a further multisensory configuration (integrated into the rearview mirror assembly as in FIG. 2B) where the printed circuit board 101is connected by plural flexible connectors 103 (in this case threeflexible connectors) to plural (in this case three) associated emitterarrays 105 each include a linear array of 6 emitters on an associateddaughter board. In the particular embodiment shown, a camera C or otherlight/radiation sensor may also be included within the sensor arrays.The FIG. 2A/2B example arrangement thus provides three TOF sensor arrays182 (one forward looking, one rearward looking, and one side looking)within a single projecting rear view mirror assembly. By providing sucha rear view mirror assembly on each side of the car, a 6-sensor arraysystem can be compactly provided in a way that is rugged, weatherproofand aesthetically pleasing while still providing a 360 degree coverageas shown in FIG. 9. The sensor technology herein is not limited tolinear arrays; other embodiments use circular arrays of emitters asshown in FIG. 3 and other shaped array configurations (e.g., triangular)are also possible.

The method that system would operate with is shown in FIG. 7. The sensorwith on-board processing captures data 2002; that for a CWTOF sensor isoutput as a radial depth map 2004. This depth map 2004 is converted bythe on-board software to a 3D point cloud 2006, which in turn, isclassified into an object of interest 2008. The object is then tracked2010. Tracking 2010 typically entails placing each subsequent objectframe location into a Kalman filter or similar. From this, the tracklocation, velocity vector and even accelerations can be output alongwith estimates of their error. These values are then run through a“probability engine” to create an estimate of collision probability anda “severity estimator” to estimate the severity of any collision, asdescribed above.

Example Sensor/System Packaging and Form Factor

The system has been designed for ease of packaging onto existing andfuture vehicles.

FIGS. 8A, 8B show example non-limiting embodiments of packaging schemesthat incorporates a CWTOF sensor 182 into the headlamps and tail lampsof a vehicle. FIG. 8A shows an arrangement including a forward-lookingsensor 2101 integrated into a headlamp arrangement of a vehicle and asideways-looking sensor 2102 integrated into a side portion of the sameheadlamp arrangement. In this particular non-limiting example, the roundsensor 2101 to the left of 3 vertical pairs of emitters in a row isfacing forward and the sidelooking sensor 2102 comprises a sidewayslooking sensor with an enclosed emitter panel. FIG. 8B meanwhile showstwo sensors (a sideways looking sensor 2202 and a rearward-lookingsensor 2201) integrated into a tail lamp. In this particular FIG. 8Bexample, sensor 2201 is a straightforward rearward-looking sensor with 3emitters above and 3 emitter below and sensor 2202 is also in the taillamp but looking sideways with a row of 6 emitters to the left and asingle sensor to the right. When configured into the headlamps and taillamps in this manner, the system can provide 360 degree protection asshown in FIG. 9. In particular, FIG. 9 shows a birds eye view of avehicle with 8 sensors (two in each of left and right tail lights asshown in FIG. 8B and two in each of left and right headlamps as shown inFIG. 8A) to provide 360 degrees of sensing with a 90 degree horizontalfield of view (FOV). These are only non-limiting examples.

The technology herein may be used in any context in which collisions mayoccur. For example, the sensor(s) may be mounted on a stationary objectthat is about to the struck by a moving object, or it/they may bemounted on a moving object that is about to strike an stationary object,or it/they may be mounted on a moving object that is about to strike orbe struck by a moving object. A countermeasure may be deployed based ontracking information such as the direction of impact, the speed ofimpact, the force of impact, the time of impact, etc. The countermeasurecan be deployed before impact, or it can be prepared for deployment andnot employed until the moment of impact or even after the moment ofimpact. Multiple countermeasures can be deployed, e.g., somecountermeasures can be deployed before impact and other countermeasurescan be deployed at impact or after impact. Countermeasures can beemployed in a “stair stepped” manner, based on severity of expectedimpact, direction of expected impact, expected force of impact, measuredforce of actual impact (once impact actually occurs), etc., as thesystem learns more information about the nature of the impact. Sensingand tracking may be performed continually—not just before impact.

All patents and publications cited herein are incorporated by referenceas if expressly set forth.

While the invention has been described in connection with what ispresently considered to be the most practical and preferred embodiment,it is to be understood that the invention is not to be limited to thedisclosed embodiment, but on the contrary, is intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims.

1. A vehicle system comprising: a sensor that senses an object; and aprocessor operatively coupled to the sensor that determines, in responseto object sensing, an imminent collision with the sensed object beforethe collision occurs.
 2. The system according to claim 1, furthercharacterized in that the processor determines an imminent collisionduring a time window within the range of one second, or one-half second,or 250 milliseconds or 200 milliseconds or 150 milliseconds or 100milliseconds or 50 milliseconds, before a collision occurs.
 3. Thesystem according to claim 1, further characterized in that the sensorcomprises a non-contact sensor or a time-of-flight sensor or astereoscopic sensor or a short-range radar sensor or an imaging acousticsensor.
 4. The system according to claim 1, further characterized inthat the sensor senses parameters relating to the range, rate of closingand direction of closing of the object.
 5. The system according to claim1, further characterized in that the sensor senses positional and/ormotion information of the vehicle relative to moving or stationaryobjects.
 6. The system according to claim 1, further characterized inthat the processor determines imminent collision once the collision isunavoidable and highly probable.
 7. The system according to claim 1,further characterized in that the processor tracks the sensed object asit closes.
 8. The system of claim 1 further characterized in that thesensing is performed continually, and a countermeasure(s) may bedeployed at least in part after the collision occurs.
 9. The system ofclaim 1, further comprising storing and forwarding the imminent or otherlikely collisions.
 10. The system of claim 1, further comprising: theprocessor determining, in response to object sensing, driving habitsthat indicate a likelihood of collision or avoidance of collision withobjects, and using a reporting device to report the determination.
 11. Avehicle system comprising: a sensor that senses closing objects; and aprocessor operatively coupled to the sensor that determines, in responseto object sensing, whether driving behavior associated with the sensedclosing objects is characteristic of whether collisions are likely andreports information relating to driving habits.
 12. A method comprising:sensing the position and/or motion of objects relative to a movingvehicle; determining, in response to object sensing, possible collisionconditions with the sensed objects will occur in time windows of onesecond or less; and reporting the determination(s).
 13. The method ofclaim 12 further characterized in that the sensing is performedcontinually while the vehicle is in operation.
 14. A vehicle systemcomprising: a sensor that senses a closing object; and a processoroperatively coupled to the sensor that determines, in response to objectsensing, whether the vehicle is being driven in a risky manner.
 15. Thesystem of claim 14 further characterized in that claim the sensor and/orprocessor is integrated in headlamp, tail lamp, rearview mirror housing,a pillar, B pillar, front grill, front bumper, rear bumper or trunkhousing.
 16. A vehicle system comprising: a sensor that senses a closingobject; and a processor operatively coupled to the sensor thatdetermines, in response to object sensing, driver attentiveness,aggressiveness, distancing, and skillfulness in terms of relations toneighboring vehicles and distinguishes between maneuvers in closeproximity to other vehicles from maneuvers not in close proximity toother vehicles.
 17. The system of claim 16 wherein the sensor comprisesa time of flight sensor.
 18. The system of claim 16 wherein theprocessor and sensor are both disposed on a common vehicle.
 19. Thesystem of claim 16 wherein the sensor is disposed on a vehicle and theprocessor is located remotely to the vehicle and is operativelyconnected to the sensor via a communications link.